Initial Voices for Science Data Analysis
Overview
The Voices for Science (VFS) program began in 2018, with data collected at four points along the one year fellowship: at the beginning of the program (“pre”), after the DC workshop (“workshop”), half-way through the program (“mid”, and at the end of the program (“post”). Participants were asked throughout the fellowship about the effectiveness of program elements, feelings about their identity as a science communicator, and feedback on the program. The stated goals of the VFS program are as follows:
Participants are engaged and mobilized long-term
Participants catalyze change for external audiences
Participants catalyze engagement in others
Participants reach new audiences in key geographic regions
Participants raise status of science policy and communications engagement within AGU
Demographics
Demographic data for gender, career stage, and location was scraped manually from the AGU sharing science website via public profiles.
Does AGU have any other demographic data that they collect on participants? If not, some basic questions could be integrated into future surveys to avoid manual data collection on gender, race, university, and/or career stage.
Gender
Gender identity was defined by pronoun usage on public profiles, with she/her pronouns defined as female, he/him as male, and they/their as non-binary. 76% of total participants in the VFS program identified as female, 23% as male, and 1% as non-binary. Given that 44% of science and engineering doctorate earners are women (NSF Survey of Earned Doctorates 2022), the VFS program over-represents women as fellows, implying a gendered difference in interest in science communication and policy training.
Participants that identify as women appear to have a preference for the communications track while those that identify as men prefer the policy track. This is especially apparent in the first two years of the program in 2018 and 2019. In 2019, the policy track had more men than women, despite women making up 68% of participants. Similarly, in 2023, no men entered the communications track.
Career Stage
Career stage was assigned as masters student, technician, PhD students postdoctoral researcher, staff scientist, or professor. In the first year, 17% of participants were professors and 33% were staff scientists, representing the largest proportion of later career participants throughout the program. On average a majority of VFS participants have been early career students: 4% were masters students, 6% were technicians, 53% were PhD students, 10% were postdocs, 15% were staff scientists and 13% were professors.
Was AGU purposefully targeting early career scientists in building the VFS program? Does career stage play any factor into acceptance or rejection from the program?
Location
U.S. Participants in the Voices for Science program have spanned 39 states plus Puerto Rico and the Virgin Islands. California has had the most participants with participants (n = 12), followed by Virginia (n=6), Florida (n=5), New York (n=5), Pennsylvania (n=5), and Washington (n=5). Overall, the program displays a wide geographical diversity, with participants from all regions of the U.S.
Did AGU target specific states for applicants? One of the stated goals of the program was to reach audiences in key geographic regions, but what were those regions defined as?
University
With a relatively small data-set (n=226), it is somewhat surprising that there are so many clear clusters on the map for participants that are coming from the same university. In total, there are 137 unique institutions - 19 of which have more than two participants over the six years of the program. The five most popular universities that have hosted participants are University of Arizona (n=11), UC Irvine (n=9), University of Alabama (n=6), Colorado State University (n=5), and Scripps Institution of Oceanography (n=5).
This implies a boost in attendance following the years a participant from a specific university does the VFS program. One student who participates can inspire others in their department community to try the program the next year. For example, Arizona State University, Caltech, North Carolina State University, University of Arizona, and UC Irvine all have spikes in attendance. This implies some success in two goals of the program: “catalyzing change in others” and “raising the status of science policy and communications engagement.”
Changes throughout the Program
Outreach Identity
Participants were asked three recurring questions about their identity as a scientist that engages in outreach activities. Participants were asked these three questions four times throughout the program. They could respond not at all, a little, a moderate amount, a lot, or a great deal. Only 2018, 2021, and 2023 have data across all four time points.
Do you consider yourself someone who does outreach?
Across all years, participants generally increased in their identity as a person who does outreach. In 2018, the percentage of participants that responded “a little” decreased from 20% to 0% from the beginning to the end of the program, while participants who responded “a lot” increased from 17% to 33%. Similar patterns were seen throughout the three years of complete data. While positive responses of “A lot” and “A great deal” increased most years throughout the program, the portion of respondents who answered “None at all” or “A little” consistently decreased.
If you consider each individual, you can categorize their responses as either increasing from pre to post program, decreasing from pre to post program, having no change throughout the program, or having a spike in an answer without an overall increase or decrease. Each individual response from 2018, 2021, and 2023 are plotted below and colored by response type. Only 4 out of 57 respondents showed a decrease, with the majority (34 out of 57) increasing in their sense of identity as a person who does outreach. A significant portion (13 out of 57) reported no change throughout the program. Four respondents, for example, remained at a response of “a great deal” throughout all checkpoints, implying that they were already at a maximum level before the program began.
Do you consider yourself someone who leads others in outreach?
The second question on outreach focuses on identity as a leader in their community. Like in question one, participants generally increased in their self-perception as an outreach leader throughout the program. This is especially evident in the decrease of those that responded “a little”: in 2018, the percentage of respondents who began at this level was 33% and decreased to 27% by the workshop, 10% by the halfway point, and 3% by the end of the program. Similarly, 41% of respondents in 2021 and 36% of respondents in 2023 responded “a little” at the pre checkpoint, but this decreased to 15% and 11%, respectively, by post assessment.
At the individual level, 34 of the 57 (~60%) participants who responded at each time point showed an increase in their identity as a person who leads others in outreach. Beyond just participating in outreach, these responses indicate that there are significant shifts for individuals in the program to see themselves as catalysts for societal change.
Do you feel like you are part of a community of scientists that do outreach?
The third question interrogates the sense of community built throughout the fellowship with the participants. Similar to the previous two outreach questions, participants generally increased in their sense of community throughout the program. Unlike previous questions, however, there was a large spike following the workshop, implying that this intervention was the strongest component of the program in fostering community among participants. In 2018, for example, the percentage of respondents who responded “a lot” or “a great deal” increased from 40 to 80% following the workshop.
At the individual level, 40 out of 57 (70%) of participants that responded at each time point showed an increase in their sense of being part of a community of scientists that do outreach. The VFS program appears to give the biggest boost to individuals in this sense of community - even more than boosts to feelings of participation in outreach or leadership in outreach. Only six participants decreased while seven had no change over the program, and four had a spike.
Audience Knowledge
Participants were asked three recurring questions regarding their perceived knowledge and comfort around communicating with their target audience. Between 2018 to 2022, respondents were asked to enter a number between 1 and 100, but in 2023, the survey format changed to a 1 through 5 scale. Only 2018, 2021, and 2023 have data across all four time points.
Why did the numerical categories change from 0 to 100 to 1 through 5? The 0 to 100 category may be better to keep for consistency moving forward and to allow a greater range of responses.
What is your knowledge level about how to contact your target audience?
Generally, participants displayed an increase in their knowledge level about contacting their target audience throughout the program, with the biggest boost in knowledge following the workshop. This knowledge increase is especially clear at the individual level, where 50 out of 57 participants with data across all time points showed an increase in their reported knowledge in contacting their target audience.
What is your knowledge level about how to effectively message to your target audience?
Similar to the pattern for contacting audience, participants increased in their knowledge of effectively messaging to their audience over the course of the program. The increase in knowledge is especially evident following the workshop, indicating that this experience was highly effective training. At the individual level, 85% of participants reported an increase in their knowledge level over the program regarding messaging to their audience effectively.
What is your comfort level about starting a conversation with your target audience?
The last question around target audience asked about participant comfort levels
Communication Frequency
Participants were asked before, half-way, and after the program how frequently they engaged with their target audience with possible answers varying between “never” to “every day.” Most (~69%) participants engage with their audience a few times a month to a few times a year, although throughout the program people generally increased their communication frequency. For example, the percentage of respondents who said they never engaged with their audience was 3% in 2018, 14% in 2021, and 21% in 2023, but for all three years this went to 0% by the mid checkpoint. At the individual level this increase is also clear, with 75% of participants increasing in their reported frequency throughout the program.
Can AGU provide the data for monthly check-ins throughout the program as well? That could be a nice dataset to track communication outputs in the program and tie them to communication frequency.
Institutional Support
Participants were asked before, half-way, and after the program about how supportive their institutions were regarding their participation in the program. Overall, institutional support has decreased throughout the program, with 80% of participants in 2018 mentioning “a lot” or “a great deal” of support by the end, compared to 42% in 2021 and 50% in 2023. The pattern over the program course also flipped from 2018, where support increased from pre to mid to post, to 2021 and 2023, where support lowered over the program. At the individual level, roughly one third of participants reported decreased levels of institutional support over the program.
